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1.0  EXECUTIVE  SUMMARY 


The  Department  of  Defense  (DoD)  is  currently  involved  in  a  number  of  unexploded  ordnance 
(UXO)  site  remediation  efforts  where  rapid  transition  of  advanced  technologies  can  save 
substantial  sums  of  money  and  significantly  expedite  the  transfer  of  lands  for  re-use.  One  of 
the  most  prominent  of  these  efforts  is  the  ongoing  UXO  cleanup  of  the  Kaho’olawe,  Hawaii, 
bombing  ranges.  The  difficulty  with  this  site  is  that  the  significant  magnetic  anomalies  from 
geologic  sources  and  near-surface  fragments  make  traditional  magnetometer-based  surveys 
impractical.  Standard  EM61  metal  detection  surveys  have  also  performed  poorly  in  these 
conditions  due  to  the  very  high  magnetic  susceptibility  response  of  basalt  and  basaltic  soils. 
As  of  March  1,  2000,  contractors  at  Kaho’olawe  had  detected  12,121  subsurface  anomalies, 
and  after  digging,  they  found  that  only  4%  are  UXO,  32%  are  false  positives  due  to  geologic 
variations,  and  64%  are  due  to  buried  metal  from  both  UXO  and  non-UXO-related  materials 
(Reference  1).  The  focus  of  this  project  was  to  evaluate  the  Geonics  EM63  multigate  time 
domain  metal  detector  to  quantify  its  detection,  discrimination,  cost,  and  production  rates. 
This  project  was  designed  to  incorporate  the  lessons  learned  from  previous  UXO  technology 
demonstrations  and  to  extend  the  results  of  the  Jefferson  Proving  Ground  (JPG)  Phase  IV 
Demonstrations  that  were  completed  during  FY  97. 

NAEVA  Geophysics  was  contacted  by  the  Environmental  Security  Technology  Certification 
Program  (ESTCP)  under  Contract  Number  200035  to  demonstrate  detection  and 
discrimination  abilities  of  Geonics’  EM63  multichannel  (multitime  gate)  instrument  through 
the  development  of  algorithms  and  decay  curve  analysis.  The  purpose  of  this  advanced 
technology  is  to  reduce  the  cost  and  time  spent  in  UXO  remediation.  To  achieve  this  goal, 
the  contract  was  divided  into  three  main  tasks.  Task  One  was  conducted  at  NAEVA’ s  office 
in  Charlottesville,  Virginia  and  the  Naval  Research  Laboratory  (NRL)  Test  Field  at  the  Army 
Research  Lab’s  Blossom  Point  Research  Facility  in  Maryland  to  establish  an  ordnance  decay 
curve  analysis  database  and  to  develop  algorithms  for  discrimination.  Task  Two,  conducted 
at  JPG  in  Madison,  Indiana,  was  a  government-sponsored  blind  test  to  detennine  the 
capabilities  of  the  EM63  and  other  electromagnetic  interference  (EMI)  instruments  under  a 
realistic  nonmagnetic  clutter  environment.  Task  Three  was  an  expansion  of  JPG  conducted 
on  Kaho’olawe,  Hawaii,  to  determine  detection  and  discrimination  capabilities  under  a  high 
magnetic  background  environment. 

The  evaluation  objectives  of  the  EM63  (and  the  other  systems)  were: 

1.  To  evaluate  detection  and  discrimination  capabilities  under  realistic  target/geologic 
clutter/man-made  clutter/topography  scenarios,  while  operating  as  efficiently  as 
possible  by  minimizing  time,  manpower,  and  costs. 

2.  To  evaluate  the  ability  to  analyze  survey  data  in  a  timely  manner  and  provide 
prioritized  “dig  lists”  with  associated  confidence  levels. 

3.  To  collect  manpower,  time,  productivity,  and  cost  data  for  all  data  acquisition  and 
processing  tasks. 


1 


4. 


To  provide  quality,  georeferenced  data  for  postdemonstration  (off-site)  analysis,  for 
development  of  receiver  operator  characteristics  (ROC)  curves,  and  for  use  by  other 
government,  university,  and  industry  researchers  to  develop  improved  analysis 
technologies. 

There  were  no  regulatory  issues  in  connection  with  NAEVA’s  ESTCP  demonstration 
performance  at  Blossom  Point,  JPG  V,  or  Kaho’olawe.  The  primary  regulatory  issue,  which 
will  affect  the  adoption  of  discrimination  technology  such  as  EM63,  will  be  gaining  the 
confidence  and  approval  of  federal,  state,  and  local  regulators,  stakeholders,  and  users. 
Acceptance  by  organizations  such  as  the  Army  Corps  of  Engineers  and  Naval  Facilities  and 
Engineering  Command  will  be  needed  so  that  future  requests  for  proposals  (RFP)  will 
include  such  innovative  technology.  This  controlled  site  ESTCP  demonstration  was  the  first 
to  employ  realistic  conditions,  which  will  allow  side-by-side  comparisons  of  discrimination 
performance,  production  rates,  and  costs.  Acceptance  of  discrimination  technology  (that  is, 
not  digging  part  of  a  prioritized  geophysical  target  list)  ultimately  requires  a  cost/risk 
evaluation  by  the  regulatory  agencies. 

Overall,  the  results  look  extremely  promising.  In  areas  of  nonmagnetic  background, 
detection  is  -92%  and  discrimination  of  ordnance  from  clutter  is  -75%.  In  areas  of  magnetic 
background,  detection  is  -61%  and  discrimination  of  ordnance  form  clutter  -61%.  This  is  an 
improvement  over  conventional  methods,  but  many  factors  influenced  these  results.  For  JPG 
V,  nonmagnetic  background  site,  a  number  of  objects  were  emplaced  below  the  detection 
threshold.  The  EM63  is  not  as  effective  in  detecting  small  items  (20  mm),  and  the  site  was 
not  completely  cleared  before  the  demonstration  began.  These  factors  also  influenced 
Kaho’olawe;  in  addition,  the  site  was  difficult  to  access. 
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2.0 


TECHNOLOGY  DEVELOPMENT 


2.1  TECHNOLOGY  DEVELOPMENT  AND  APPLICATION 

The  EM63  has  been  modified  twice  in  the  last  3  years  of  this  contract  by  the  manufacturer, 
Geonics  Limited,  of  Toronto,  Canada.  It  generates  a  pulsed  (time  domain)  primary  magnetic 
field  (using  a  horizontal,  multiturn,  air  cored,  1  m  x  1  m  transmitter  coil  40  cm  above  the 
ground  surface)  that  induces  Faraday  eddy  currents  and  magnetic  polarization  in  nearby 
metallic  and  ferromagnetic  objects.  The  decay  of  the  resulting  secondary  magnetic  fields 
over  time  is  detected  in  receiver  coils  40  cm  (bottom  coil)  and  80  cm  above  the  ground 
(coaxial  with  the  transmitter  coil).  The  observed  decay  as  a  function  of  time  is  determined 
by  the  character  of  the  target  object  (size,  shape,  orientation,  and  composition).  In  general, 
the  observed  decay  is  a  linear  superposition  of  the  axial  (longitudinal)  and  transverse 
excitation  responses  of  the  target  object.  The  transmitter  current  waveform  is  bipolar 
rectangular  with  25%  duty  cycle,  15  amps  maximum.  The  system  controller  is  a  PR04000 
field  computer  (486  AMD  processor).  The  data  acquisition  (DAQ)  dynamic  range  is  18  bits. 
Acquisition  speed  is  six  records  (25  time  gates  per  record)  per  second.  Using  the  above- 
mentioned  basic  configuration,  the  main  receiver,  transmitter  coils,  and  data  acquisition 
computer  remained  the  same. 

2.1.1  Blossom  Point  Configuration  and  JPG  Coil  Configuration 

The  electromagnetic  (EM)  bottom  sensor  coil  is  a  circular  50  cm  diameter  multiturn,  air- 
cored  coil,  coplanar  with  the  transmitter  coil,  with  500k  Hz  bandwidth.  The  top  sensor  coil  is 
a  1  m  x  1  m  square  coil  40  cm  above  the  bottom  coil  and  transmitter  coil  (identical  to  the 
EM61  top  coil).  Twenty  to  30  geometrically  spaced  time  gates  are  measured,  covering  a 
range  from  180  microseconds  to  20  milliseconds  (medium  base  frequency)  or  180 
microseconds  to  7  milliseconds  (high  base  frequency).  (See  Figure  1.) 
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Figure  1.  Configuration  of  Coils  During  Blossom  Point  and  JPG  V  Demonstration. 


2.1.2  Kaho’olawe  Coil  Configuration 

The  EM  bottom  receiver  coil  is  a  square  0.5  m  x  0.5  m  multiturn,  air-cored  coil,  coplanar 
with  the  1  m  x  1  m  transmitter  coil.  The  top  receiver  coil,  also  a  square  0.5  m  x  0.5  m 
multiturn,  air-cored  coil,  is  located  60  cm  above  the  bottom  coil.  In  between  (30  cm  from  the 
bottom  coil)  the  two  0.5  m  x  0.5  m  multiturn,  air-cored  coils,  is  a  third  0.5  m  x  0.5  m 
multiturn,  air-cored  calibration  coil.  The  second,  lmxlm  square  coil,  is  40  cm  above  the 
bottom  coil  and  transmitter  coil  (identical  to  the  EM61  top  coil).  However,  this  coil  acts  only 
as  a  stabilizing  coil;  it  is  not  a  receiver  coil  as  in  the  previous  model  used  for  JPG  V. 
Twenty-six  geometrically  spaced  time  gates  are  measured,  covering  a  range  from  180 
microseconds  to  25  milliseconds.  (See  Figure  2.) 

2.2  PROCESS  DESCRIPTION 

Refer  to  Section  3  and  Section  4. 
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Figure  2.  Configuration  of  EM63  During  the  Kaho’olawe  Demonstration. 


2.3  PREVIOUS  TESTING  OF  THE  TECHNOLOGY 

NAEVA  Geophysics  demonstrated  the  use  of  the  Geonics  Protem  time  domain  EM  system 
for  UXO  discrimination  at  the  Advanced  UXO  Detection/Discrimination  Technology 
Demonstration  at  the  JPG  in  1998.  This  system  was  the  prototype  for  the  new  EM63 
multitime  gate  system  that  became  available  late  in  1999. 

2.4  ADVANTAGES  AND  LIMITATIONS  OF  THE  TECHNOLOGY 

The  EM63  multichannel/multitime  gate  information  permits  discrimination  of  various 
metallic  ordnance  items  from  metallic  nonordnance  items  and  may  also  discriminate 
nonmetallic  materials  (i.e.,  basalt  and  basalt  soils)  from  metallic  objects,  using  the  shape  of 
time  decay  response  across  the  instrument’s  20-30  time  gates. 

Time  decay  curve  shape  analysis  permits  the  recognition  of  specific  ordnance  items  that  have 
been  tested  and  cataloged  in  a  database.  It  does  not  permit  generic  discrimination  of 
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ordnance  from  nonordnance  by  class.  Some  nonordnance  items  may,  by  chance,  exhibit 
decay  curves  that  match  certain  ordnance  items.  Therefore,  the  list  of  ordnance  items  to  be 
recognized  should  be  restricted  to  those  actually  expected  on  each  particular  remediation  site. 
The  EM63  technology  is  still  in  development  and  has  been  modified  twice  in  the  last  3  years. 
The  decay  curve  analysis  library  must  be  reestablished  and  the  algorithms  modified  with 
each  modification. 
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3.0  DEMONSTRATION  DESIGN 


3.1  PERFORMANCE  OBJECTIVES 

NAEVA’s  primary  technical  objective  was  to  determine  the  detection  and  discrimination 
capabilities  of  the  EM63  multichannel,  time-domain  EMI  system  in  nonmagnetic  and 
magnetic  clutter  environments. 

3.2  SELECTION  OF  TEST  SITE 

The  test  site  selections  were  done  by  ESTCP. 

3.3  TEST  SITE/FACILITY  HISTORY/CHARACTERISTICS 

3.3.1  Blossom  Point 

NRL  Test  Field  at  the  Army  Research  Lab’s  Blossom  Point  Facility  is  a  1,600-acre  U.S. 
Army  installation  near  LaPlata,  Maryland,  in  southern  Charles  County.  The  Blossom  Point 
Field  Test  Facility  is  an  active  facility  under  the  U.S.  Army  Research  Laboratory  in  Adelphi, 
Maryland.  During  World  War  II,  Harry  Diamond  and  his  team  at  the  National  Bureau  of 
Standards  (NBS),  now  named  the  National  Institute  of  Standards  and  Technology  (NIST) 
needed  open  areas  where  they  could  test  the  fuses  they  were  developing.  They  established 
test  sites  at  Aberdeen  Proving  Ground,  Maryland;  Fort  Fisher,  North  Carolina;  and,  in  early 
1943,  NBS  leased  land  and  established  a  proving  ground  for  proximity  fuses  at  Blossom 
Point.  By  September  1945,  14,000  rocket  and  mortar  rounds  had  been  fired.  In  1953,  the 
lease  on  the  property  was  transferred  to  the  Army,  which  operated  the  property  as  a  fast- 
reaction,  low-cost  range  for  experimental  work.  Firing  ranges  provided  a  2,000-yard 
maximum  range  for  land  impact  and  a  10,000-yard  maximum  for  water  impact.  During  the 
Vietnam  War,  the  Anny’s  Harry  Diamond  Laboratory  was  very  active  at  the  site. 

3.3.2  Jefferson  Proving  Ground  (JPG  V) 

Inert  UXO  and  natural  and  manmade  clutter  items  were  emplaced  at  the  three  controlled 
grids  at  JPG  for  demonstrators  to  test  their  detection  and  discrimination  capabilities  under 
realistic  conditions  and  to  allow  the  government  to  estimate  production  and  cost  rates  in 
actual  cleanup  operations.  The  three  1 -hectare  test  grids  were  chosen  to  provide  grids 
characterized  as  relatively  low,  medium,  and  high  magnetic  clutter  (from  geologic  sources). 
Figure  3  shows  the  locations  of  these  three  demonstration  grids  within  the  JPG  16-hectare 
(40  acre)  and  Waterways  Experiment  Station  (WES)  test  sites.  Grid  1  contains  an  elongate 
high  magnetic  anomaly  (+150  nT  to  -100  nT)  and  was  seeded  with  the  largest  concentration 
of  inert  UXO  targets  and  clutter  items.  Grid  2  exhibits  a  more  moderate  magnetic  response 
(0  to  35  nT)  and  irregular  topography.  Grid  3  contained  very  low  magnetic  terrain  response 
and  very  flat  topographic  relief.  It  was  seeded  with  the  fewest  targets  and  clutter  items.  It 
should  be  noted  that  all  of  these  grids  are  presumably  low  magnetic  relief  compared  to 
Hawaiian  basaltic  terrains,  which  are  normally  +/-  many  thousands  of  nT  due  to  the  very 
high  magnetic  susceptibilities  and  magnetic  homogeneity  of  basalt  and  basaltic  soils. 
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Figure  3.  Site  Map  of  JPG  V  Grids. 


Plastic  flags  were  placed  around  the  perimeters  of  the  three  test  grids  (oriented  to  magnetic 
north),  and  survey  control  points  were  available  on  or  beside  all  three  grids.  Twelve  inert, 
cleaned,  and  degaussed  ordnance  types,  ranging  in  size  from  20  mm  to  155  mm,  were 
emplaced  on  the  three  test  grids,  together  with  representative  nonordnance  (clutter)  items  and 
basaltic  samples. 

3.3.3  Kaho’olawe,  Hawaii 

Kaho’olawe  Island  consists  of  the  summit  of  a  single  volcanic  dome  that  reaches  a  peak 
elevation  of  1,477  ft  above  mean  sea  level.  The  island  was  used  as  a  weapons  range  and 
military  training  area  from  1941  until  1990.  In  1993,  Title  X  of  the  FY  1994  Department  of 
Defense  Appropriations  Act  directed  the  cleanup  of  ranges  in  Kaho’olawe  Island.  Title  X 
allocated  $400  million  for  UXO  remediation  that  began  in  1993.  As  of  August  2000,  only 
1,100  acres  of  the  18,000  acres  requiring  remediation  had  been  completed.  (Reference  2.) 

The  overall  set-up  for  the  exercise  is  described  in  the  Advanced  UXO  Detection/ 
Discrimination  Technology  Demonstration  Plan  (Reference  5)  by  the  Naval  Explosive 
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Ordnance  Disposal  Technology  Division  (NAVEODTECHDIV).  Multiple  samples  of  20  to 
30  inert  ordnance  types  and  a  variety  of  nonordnance  and  clutter  items  were  emplaced  on 
three  contiguous  demonstration  grids  (A,  B,  and  C),  which  totaled  about  2  hectares.  Figure  4 
shows  the  distribution  of  these  demonstration  grids  (Grid  A,  composed  of  four  30  m  x  30  m 
subgrids;  Grid  B,  composed  of  nine  30  m  x  30  m  subgrids  and  three  20  m  x  30  m  subgrids; 
and  Grid  C,  composed  of  six  30  m  x  30  m  subgrids). 


Figure  4.  Site  Map  of  Kaho’olawe  Demonstration  Area. 


3.4  SAMPLING/MONITORING  PROCEDURES 


3.4.1  NAEVA-Geophysical  Associates  (GPA)  Data  Quality  Control  Plans 


The  EM63  was  static  tested  for  zero  calibration 
and  instrument  (plus  ambient)  noise  at  the 
beginning  of  each  survey  lane  file.  The  first  line 
was  repeated  (bidirectional)  to  verify  amplitude 
and  location  repeatability.  As  soon  as  the  file  was 
complete,  it  was  checked  for  data  gaps  and/or 
poor  global  positioning  system  (GPS)  position 
recovery,  and  portions  were  repeated,  if  necessary 
(generally  due  to  poor  satellite  availability).  The 
repeatability  of  the  first  line  in  each  grid  lane  file 
(and  the  amplitude  response  of  the  calibration 
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sphere)  was  also  verified,  and  terrain  noise  was  Figure  5.  Check  Ashtech  GPS 
inspected.  Figure  5  shows  GPS  position  checks  Positions,  Lane  B18-28. 
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for  a  problematic  lane  file  (lane  B 18-28,  tree  at  northeast  end);  black  denotes  GPS  first 
quality  “fix,”  while  red  denotes  GPS  second  quality  “float.”  Float  positions  were  sometimes, 
but  not  always,  usable. 


The  EM63  was  operated  on  a  nonmetallic  test  table 
in  static  mode  at  the  beginning  and  end  of  each  lane 
file  to  zero  the  instrument  (away  from  possible 
background  response)  before  each  survey  period  and 
to  check  for  calibration  drift  after  each  survey  period. 
Figure  6  illustrates  EM63  zeroing  in  air  before  grid 
lane  surveying. 

A  standard  3.5  in  iron  calibration  sphere  was  placed 
at  zero  depth  just  north  of  the  north  end  of  the  first 
survey  line  in  each  lane,  in  order  to  verify  stable 
amplitude  response.  The  initial  line  in  each  lane  file 
was  surveyed  in  southeast  and  northwest  directions 
to  verify  data  repeatability  and  satisfactory  positional 
latency  (lag)  corrections.  Data  was  measured  in 

Surveying. 

narrow  blocks  or  lanes  approximately  10 
m  wide  over  the  full  northwest-southeast 
extent  of  each  grid  (or  contiguous  group  of 
grids).  This  was  done  due  to  memory 
limitations  in  the  EM63  and  to  avoid 
longer  term  zero  calibration  drift  (data 
acquisition  required  approximately  30  to 
60  minutes  per  10  m  lane).  Figure  7 
illustrates  surveying  with  the  EM63. 

Northeast-southwest  guide  ropes  (parallel  to  Demo  Grid  B  with  the  EM63  (ropes  at 
prevailing  wind)  were  spaced  2  m  apart  to  2  m  intervals), 

ensure  straight  survey  lines  with  0.5  m-line 
spacing. 

3.4.2.2  Blossom  Point  Data  Acquisition  Procedures 

NAEVA  made  Geonics  EM63  field  measurements  at  Blossom  Point  with  new  software 
developed  by  NAEVA,  GPA,  and  GEONICS  that  acquired,  integrated,  and  lag-corrected 
real-time  kinematic  (RTK)  differential  GPS  for  data  location  over  NRL’s  original  test  plot. 
The  test  plot,  approximately  30  m  x  100  m,  contained  five  north-south  lanes  along  which  a 


3.4.2  Data  Acquisition  Procedures 
3.4.2. 1  General 


Figure  6.  Zeroing  the  EM63  Before 
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variety  of  target  objects  were  emplaced  on  6-foot  centers.  A  small  plot,  lane  Y,  with  very 
small  ordnance  items  (50  cal,  20  mm,  and  30  mm)  was  added  for  NAEVA’s  exercise. 
NAEVA  also  temporarily  placed  a  3.5-inch  diameter  iron  sphere  at  zero  depth  at  the  north 
end  of  each  target  lane  for  calibration  purposes.  In  addition,  tapes  were  laid  out  to  help  the 
instrument  operator  walk  straight  lines  (not  necessary  with  GPS,  but  helpful  to  ensure 
uniform  coverage  of  the  surveyed  area). 

3.4.2.3  JPG  V  Data  Acquisition  Procedures 

EM-63  data  was  acquired  on  the  three  1 -hectare  test  grids,  starting  with  Grid  3,  which  was 
conveniently  located  (near  the  trailer  and  a  GPS  reference  monument)  and  posed  the  least 
topographic  problems  (area  was  relatively  smooth,  dry,  and  level,  with  few  trees  and  other 
obstructions).  Data  was  measured  in  narrow  blocks  or  lanes  10  m  wide,  over  the  full  100  m 
north-south  extent  of  each  grid.  This  was  done  due  to  memory  limitations  in  the  EM-63,  and 
to  avoid  longer  term  zero  calibration  drift  (approximately  40  to  60  minutes  per  10  m  lane). 
North-south  ropes  were  spaced  2  m  apart  to  ensure  straight  survey  lines  with  a  0.5  m  line 
spacing.  Each  lane  was  numbered  in  order  from  west  to  east  (3-1,  3-2,  and  so  forth).  Each 
raw  (binary)  lane  file  contains  approximately  2  Mb  of  data.  When  necessitated  by  GPS  or 
other  data  problems,  repeat  lane  files  were  measured,  and  named  3-2b,  3-2c,  and  so  forth. 
GPS  positions  were  acquired  at  a  rate  of  1  per  second,  and  EM-63  readings  were  collected  at 
a  rate  of  5  per  second,  yielding  a  data  density  of  one  reading  approximately  every  20  cm. 

3.4.2.4  Kaho’olawe  Data  Acquisition  Procedures 

In  accordance  with  instructions  for  Kaho’olawe,  the  demonstration  grids  were  surveyed  in 
three  blocks,  which  were  designated  A  (the  northernmost  four  30  m  x  30  m  grids),  B  (the  1 
hectare  grid),  and  C  (the  last  six  30  m  x  30  m  grids)  (see  Figure  4).  Data  was  acquired  in 
northeast-southwest  lanes,  generally  10  m  wide.  Each  lane  was  numbered  in  order  from 
southeast  to  northwest  (A0- 10,  A 10-20,  A20-30,  etc.).  Each  raw  (binary  EM63-GPS 
composite)  lane  file  contains  approximately  1  to  2  Mb  of  data.  When  necessitated  by  GPS  or 
other  data  problems,  repeat  lane  files  were  measured  and  named  A10-20b,  etc.  GPS 
positions  were  acquired  at  a  rate  of  1  per  second,  and  EM63  readings  were  collected  at  a  rate 
of  5  per  second,  yielding  a  data  density  of  one  reading  approximately  every  10  to  20  cm. 

3.5  DATA  ANALYSIS  PROCEDURES 

The  basic  EM63  data  processing  and  analysis  steps  (as  practiced  at  the  JPG  V  demonstration 
in  2000)  are  as  follows. 

1.  GPS  checks  -  GPS  position  integration  (interpolation,  latency  corrections). 

2.  Autoleveling  (all  gates)  -  To  remove  decaying  background  response  and  calibration 
drift  across  all  time  gates.  (This  was  not  possible  at  Kaho’olawe.) 

3.  Visual  Inspection  (profiles  and  plan  contour  maps)  and  Editing  -  To  remove  bad 
data  points,  recognize  data  gaps,  cut  outside  the  grid,  and  split  lines  for  Geosoft. 
Repeat  DAQ  if  necessary.  Cropping  to  0.5  m  outside  the  grid  boundaries. 
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4. 


Target  Picking  -  Selection  of  all  targets  over  an  appropriate  amplitude  response 
threshold  established  by  yield  curve  or  data  frequency  distribution  analysis.  At 
Kaho’olawe,  it  was  possible  (and  necessary)  to  discriminate  basaltic  decay  response 
before  target  picking.  Harvest  selected  decay  curves  for  discrimination  analysis. 

5.  Comparison  of  Decay  Curves  -  From  targets  and  bench  calibration  tests  for 
expected  ordnance  items,  computation  of  chi-squared  measure  of  misfit. 

6.  Prioritization  of  target  list  -  In  order  of  increasing  chi-squared  misfit. 

Autoleveling  and  target  picking  are  usually  the  most  important  and  difficult  data  processing 
steps.  However,  at  Kaho’olawe,  the  combination  of  excessive  instrument  drift  and  variable 
basalt  background  response  made  autoleveling  impossible.  Details  of  the  alternative  EM63 
data  processing  and  analysis  steps  as  practiced  at  Kaho’olawe  are  described  below. 

In  order  to  remove  the  effects  of  the  instrumental  drift  from  the  data  and  allow  for  variable 
basalt  background  decay  response,  a  multistep  procedure  was  used.  First,  the  average  decay 
curve  of  all  points  in  a  window  (within  30  seconds  and  20  m)  was  subtracted  from  each 
decay  curve  in  that  window.  These  adjusted  decay  curves  were  then  fit  with  the  baseline 
basalt  decay,  producing  an  y“  value,  measuring  the  degree  of  misfit.  The  measurements  were 
sorted  by  y  ,  and  the  lowest  75%  were  re-averaged,  producing  a  nominal  drift  baseline 
without  the  influence  of  measured  anomalies.  This  baseline  was  then  subtracted  from  the 
data  points  in  the  center  10  m  of  the  window  to  produce  the  de-drifted  data  set,  the  windows 
were  advanced  by  10  m,  and  the  procedure  was  repeated. 

This  de-drifting  produced  a  data  set  with  both  positive  and  negative  basalt  response,  the 
former  where  the  basalt  level  was  higher  than  the  local  average,  the  latter  where  it  was  lower. 
Thus,  the  subsequent  fits  that  include  a  basalt  component  allow  that  component  to  be  either 
positive  or  negative. 

The  de-drifted  data  was  fit  with  a  single  basalt  component  first,  producing  an  y2  map  with 
peaks  where  the  instrument  response  was  non-basalt-like.  These  peaks  were  picked  with 
standard  Geosoft  software,  and  were  then  turned  into  target  lists.  The  discrimination  of  the 
targets  was  performed  by  simultaneously  fitting  both  a  basalt  component  and  components 
from  the  ordnance  library  decay  curves  to  the  decay  curve  of  the  anomaly.  This  produced  an 
y  for  each  type  of  ordnance  in  the  library.  These  y“  values  were  subtracted  from  the  basalt- 
only  y2,  and  these  Ay2  values  (corresponding  to  log-likelihood  ratios  in  a  model  comparison 
test)  were  used  to  rank  each  target. 
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4.0  PERFORMANCE  ASSESSMENT 


4.1  INTRODUCTION 

NAEVA  Geophysics  was  contacted  by  ESTCP  under  Contract  Number  200035  to  demonstrate 
detection  and  discrimination  abilities  of  Geonics  EM63  multichannel  (multitime  gate)  instrument 
through  the  development  of  algorithms  and  decay  curve  analysis.  The  purpose  of  this  advanced 
technology  is  to  reduce  the  cost  and  time  spent  in  UXO  remediation.  To  achieve  this  goal,  the 
above-mentioned  contact  was  divided  into  three  main  tasks.  Task  one,  conducted  at  NAEVA’s 
office  in  Charlottesville,  Virginia,  and  at  the  NRL  Test  Field  at  the  Army  Research  Lab’s 
Blossom  Point  Research  Facility  in  Maryland,  was  to  establish  an  ordnance  decay  curve  analysis 
database  and  the  development  of  algorithms  for  discrimination.  The  second  task,  conducted  at 
JPG  in  Madison,  Indiana,  was  a  government-sponsored  blind  test  to  detennine  the  capabilities  of 
the  EM63  and  other  EMI  instruments  under  a  realistic  nonmagnetic  clutter  environment.  Third, 
an  expansion  of  JPG  was  conducted  on  Kaho’olawe,  Hawaii,  to  determine  detection  and 
discrimination  capabilities  under  a  high  magnetic  background  environment. 

4.2  BLOSSOM  POINT 

The  first  set  of  test  measurements  using  Geonics  EM63  in  conjunction  with  EMFIT  decay-curve 
analysis  software  was  performed  at  NRL  Test  Field  at  the  Army’s  Research  Lab’s  Blossom  Point 
Research  Facility. 

4.2.1  Objectives 

The  primary  goal  of  this  test  was  to  acquire  decay  curves  from  a  variety  of  objects  and  to  verify 
that  the  EM63  produced  decay  curves  similar  to  those  observed  by  NAEVA  with  the  PROTEM 
receiver  at  JPG-IV.  The  second  goal  was  to  use  an  RTK  GPS  with  the  EM63  and  write  software 
that  could  integrate  the  GPS  readings  with  those  of  the  EM63.  The  third  and  final  goal  at 
Blossom  Point  was  to  detennine  what  discrimination  of  objects  could  be  perfonned  with  the 
EM63  decay  curve  data,  using  EMFIT  algorithms  and  software  developed  by  G.  H.  Ware,  H.  A. 
Ware,  and  W.  F.  Tompkins  of  GPA. 

4.2.2  Results/Analysis/Conclusion 

4.2.2. 1  First  Objective:  Time  Decay  Curves 

Figure  8  shows  decay  curves  from  a  variety  of  inert  ordnance  samples  taken  at  Blossom  Point. 
The  sample  ordnance  and  simulant  objects  were  rotated  at  a  depth  of  0.5  m  under  the  EM63  to 
yield  the  decay  curves  of  each  object.  Horizontal  orientation  nonnally  produces  a  lower 
amplitude  response  decay  curve  and  vertical  orientation  produces  higher  amplitude  response 
decay  curves.  Object  P  (BDU-33)  has  an  unusual  response.  Instead  of  producing  only  one 
horizontal  response,  the  BDU-33  produces  two  distinct  decay  curve  responses  near  the  horizontal 
orientation.  For  that  reason,  three  black  decay  curves  for  item  P  are  represented  in  Figure  8.  The 
characteristic  decay  curves  agree  well  with  the  PROTEM  curves  generated  during  JPG-IV.  It 
can  be  seen  from  the  figure  that  different  types  of  objects  produce  distinct  curves,  which  can  be 
used  to  classify  the  objects. 
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EM63  Time  Decay  Curves,  Blossom  Point  Bench  Test  50mmp 
(Horizontal  and  Vertical,  d  =  50cm) 

Red  =  3.5"  sphere 

Blue  =  M  (M23,  11b.  practice  bomb) 

Green  =  R  (60mm  mortar) 

Cyan  =  N  (81mm  mortar) 

Black  =  P  (BDU-33,  251b.  bomb) 

Figure  8.  Decay  Curves  for  a  Variety  of  Inert  Ordnance  Items  Spun  at  Blossom  Point. 


Figure  9  shows  decay  curves  from  lane  A  targets.  The  16  lb  sphere  response  is  red  (with  small 
circles).  The  clutter  objects  are  shown  in  yellow.  EMFIT  algorithms  and  software  attempt  to 
prioritize  a  target  list  by  testing  the  statistical  fit  of  target  decay  curves  to  the  known  decay 
curves  of  ordnance  samples  (spun  under  the  instrument  at  all  orientations). 
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EM63  Decay  Curves,  Lane  A,  Blossom  Point;  red=ord&cylinders,  blue=plates,  green=clutter 
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EM63  Time  Decay  Curves,  Blossom  Point,  Lane  A  Targets 
Red  (R)  =  ordnance  &  cylinders  (dashed) 

Red  with  circles  =  16  lb.  sphere 
Blue  (B)  =  plates 
Green  (G)  =  clutter  items 

Figure  9.  Decay  Curves  for  Lane  A,  Blossom  Point. 


4.2.2.2  Second  Objective:  Implementation  of  EM63  with  GPS 

NAEVA  made  Geonics  EM63  field  measurements  at  Blossom  Point  from  May  30  to  June  2, 
2002.  New  software  developed  by  NAEVA,  GPA,  and  GEONICS  was  used  to  acquire, 
integrate,  and  lag-correct  RTK  differential  Trimble  GPS  for  data  location.  Figures  10  and  11 
display  the  spatial  distribution  of  early  gate  3  (better  for  small  objects)  and  later  gate  10  (which 
approximates  the  EM61  time  gate)  from  the  EM63  data  set  over  NRL’s  test  plot  which  contained 
22  inert  ordnance,  22  cylinders,  13  plates,  15  clutter  items,  and  one  16  lb  spherical  shot-put. 
NAEVA  also  temporarily  placed  a  3. 5 -inch  diameter  iron  sphere  at  zero  depth  at  the  north  end  of 
each  target  lane,  for  calibration  purposes. 
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Figure  10.  EM63  Gate  10  Test  Grid,  Blossom  Point. 
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Area  of  Investigation 
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Date  of  Suivey:  May  30-31  &June  1-2.  2000 


Figure  11.  EM63  Gate  3  Test  Grid,  Blossom  Point. 


It  can  be  seen  that  all  test  objects  were  detected  at  a  5  mv  threshold,  at  least  in  the  early  gate, 
with  the  exception  of  four  relatively  deep  (74-101  cm)  60  mm  projectiles  at  Z3,  Z4,  Z5,  and  Z6. 
The  regular  shapes  of  the  spatial  anomalies  indicate  that  the  lagged  GPS  locations  were 
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successful.  If  a  comparison  of  the  contour  plots  and  the  site  layout  is  made,  a  few  easily 
explained  discrepancies  should  be  noted.  As  mentioned,  a  calibration  sphere  was  put 
approximately  10  m  north  of  each  line,  resulting  in  the  six  uniformly  sized  anomalies  at  the  top 
of  each  target  lane.  In  addition,  tapes  were  laid  out  to  help  the  instrument  operator  walk  straight 
lines  (not  necessary  with  GPS,  but  helpful  to  ensure  uniform  coverage  of  the  surveyed  area). 
Those  tapes  had  small  metallic  handles  that  were  placed  about  5  m  north  of  each  lane.  This 
resulted  in  slight  anomalies  in  the  early  gates  at  those  positions. 

Figure  12  shows  EM63  profiles  run  north  and  south  along  lane  A.  The  alignment  of  these  spatial 
anomalies  also  shows  that  correctly  lagged  GPS  locations  were  achieved.  Note  the  double 
peaked  response  of  shallow,  horizontal  targets. 


Blossom  Point,  EM63,  BPA  line  0,  Gates  1:4:21  (un-leveled) 


Figure  12.  EM63  North-South  Profile  of  Lane  A,  Blossom  Point. 


Figure  13  shows  a  raw  (unleveled)  EM63  profile  along  lane  Z  (the  instrument  was  zeroed  before 
data  acquisition,  but  drifted).  Figure  14  shows  the  same  profiles  after  autoleveling.  Clearly, 
auto  leveling  is  necessary  before  spatial  or  decay  curve  analysis  can  be  done.  Figure  14  also 
shows  the  essential  absence  of  response  to  the  deep  60  mm  targets  at  Z4,  Z5,  and  Z6.  Note  the 
calibration  sphere  response  at  the  north  end  of  each  profile.  All  of  these  profiles  display  gates  1, 
5,  9,  13,  17,  and  21  only  (omitting  the  rest  for  clarity). 
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Blossom  Point,  EM63,  BPZ  line  0,  Gates  1 :4:21  (un-leveled) 
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Figure  13.  Raw  (Unleveled)  Profile  for  Lane  Z,  Blossom  Point. 

Blossom  Point,  EM63,  BPZ  line  0,  Gates  1 :4:21  (auto-leveled) 


Figure  14.  Autoleveled  Profile  for  Lane  Z,  Blossom  Point. 

(same  profile  as  above,  except  autoleveled) 
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4.2.2.3  Third  Objective:  Decay  Fit  and  Prioritized  Target  List 


Five  classes  of  objects  are  emplaced  in  the  Blossom  Point  grid  -  ordnance,  spheres,  cylinders, 
plates,  and  clutter.  It  is  possible  to  attempt  discrimination  between  any  of  these  classes.  Table  1 
displays  all  the  emplaced  ordnance  and  clutter  items,  arranged  in  order  of  statistical  goodness  of 
fit  to  the  given  ordnance  items  (NRL  code  letters  M,  N,  P,  and  R)  by  the  EMFIT  software.  M,  N, 
P,  and  R  are  identified  in  Figure  8  and  clutter  items  are  number  Q1-Q15.  Decay  curve  samples 
were  obtained  from  bench  tests  for  M,  N,  P  and  R.  All  blossom  point  ordnance  and  clutter  items 
(identified  by  Blossom  Point  lane  and  location  number)  were  analyzed. 

As  already  mentioned,  items  3-6  in  lane  Z  are  relatively  deep  60  mm  mortars  and  were  not 
detectable  at  5  mV  threshold  in  gates  3  or  10.  Item  Z3  is  barely  detectable  at  a  lower  threshold 
(see  Figure  14).  Decay  curves  samples  from  locations  of  items  Z3,  Z4,  Z5,  and  Z6  were 
subjected  to  chi-squared  analysis.  Item  Z3  was  correctly  identified  as  a  60  mm  mortar  (NRL 
code  R).  Locations  Z4,  Z5,  and  Z6  were  pure  noise  but  gave  relatively  good  chi-squared  fits  to 
very  low  amplitude  M  or  P  (NRL  code).  This  problem  was  recognized  and  addressed  in 
subsequent  analysis  and  does  not  occur  when  only  detectable  targets  are  analyzed. 
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This  prioritized  list  is  also  displayed  as 
an  ROC  curve  in  Figure  15.  A 
satisfactory  ROC  curve  was  achieved. 
Only  one  clutter  item,  the  horseshoe, 
appears  high  in  the  list  (evidently,  it 
has  an  “ordnance-like”  decay  curve). 
The  prioritized  list  hits  100%  true 
positives  at  40%  false  positives 
(clutter). 

4.3  JPG  V 

4.3.1  Objectives 

The  overall  technical  objective  of  this 
demonstration  was  to  evaluate  the 
detection  and  discrimination 
capabilities  (including  production  rates 
and  costs)  of  the  Geonics  EM63 
multitime  gate  electromagnetic  metal 
detector  and  associated  decay  curve, 
matching  algorithms  in  realistic  clutter 
environments.  Three  test  grids  within 
JPG  were  prepared  to  represent  a  range 
of  conditions  in  order  to  identify 
relative  strengths  and  weaknesses. 


Figure  15.  Blossom  Point  ROC  Curve. 
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Table  1.  Blossom  Point  Prioritized  Target  List  EMFIT, 


Blossom  Point  Lane 

Best-Fit  Ordnance 

and  Location 

NRL 

Depth  (cm)  and 

Chi-Squared 

Number 

Code 

Type  (NRL  code) 

Misfit 

Z5 

R 

50pl 

0.78 

Z3 

R 

5  Or 

1.08 

Z1 

R 

25r 

1.3 

A4 

N 

50n 

1.74 

Z10 

R 

50n 

1.78 

Z6 

R 

5  Op 

1.8 

B2 

M 

25m 

1.8 

B3 

N 

50n 

1.98 

Z7 

R 

25r 

2.02 

C9 

Q12 

25m 

2.33 

Z4 

R 

50m 

2.54 

Z12 

R 

50m 

2.82 

Z8 

R 

25r 

2.92 

Z2 

R 

5  Op 

3.1 

E2 

P 

50n 

3.45 

BIO 

M 

25m 

4.23 

Zll 

R 

5  Or 

5.34 

E9 

Q6 

5  Or 

6.49 

A1 

M 

50n 

6.52 

B5 

Q14 

25r 

7.93 

CIO 

Q9 

25r 

8.01 

E8 

Q15 

50n 

10.5 

B14 

M 

25m 

14.5 

Z9 

R 

5  Or 

19 

D13 

N 

50n 

30.7 

El 

Q13 

50n 

36.6 

D5 

M 

25m 

67.1 

A13 

P 

5  Op 

94.3 

C2 

Qll 

25m 

191 

D1 

Q5 

25m 

201 

C3 

Qi 

5  Op 

135 

D12 

Q7 

5  Or 

253 

E3 

Q2 

5  Or 

768 

All 

Q4 

25m 

930 

D8 

Q10 

25m 

1.17E+03 

A6 

Q3 

25m 

2.31E+03 

A2 

Q8 

25m 

2.98E+03 

Nonordnance 

Items 

(XXXXX) 


xxxxx 


XXXXX 

xxxxx 

xxxxx 

xxxxx 


xxxxx 


xxxxx 

xxxxx 

xxxxx 

xxxxx 

xxxxx 

xxxxx 

xxxxx 

xxxxx 

xxxxx 


The  evaluation  objectives  for  the  JPG  controlled  site  demonstration  of  the  EM63  were: 

•  To  evaluate  detection  and  discrimination  capabilities  by  means  of  the  three  1 -hectare 
surveys  at  JPG  under  realistic  target/clutter  scenarios  and,  while  operating  efficiently,  to 
minimize  time  and  costs. 

•  To  evaluate  ability  to  analyze  data  on  site  (NAEVA-GPA  did  not  have  on-site 
processing)  and  provide  prioritized  target  lists. 

•  To  collect  manpower,  time,  productivity,  and  cost  data  for  all  data  acquisition  and 
processing  tasks. 

•  To  compare  the  performance  of  the  Geonics  EM63  and  other  advanced,  demonstrated 
technologies  with  the  baseline  magnetic  gradiometer  and  flag  technology. 

•  To  provide  quality,  georeferenced  data  for  postdemonstration  (off-site)  analysis,  for 
development  of  ROC  curves,  and  for  use  by  other  government,  university,  and  industry 
researchers  to  develop  improved  analysis  technologies. 

4.3.2  Overall  Performance  Results 

NAEVA’s  perfonnance  results  for  Grids  1,  2,  and  3  are  best  summarized  by  ROC  curves 
generated  from  the  initial  (on-site)  and  subsequent  (off-site)  prioritized  target  lists  (with  and 
without  20  mm).  Figure  16  displays  the  on-site  (with  20  mm)  ROC  curves  and  the  Percent 
Detected  (Pd)  versus  False  Alarm  Count  (FAC)  points  for  baseline  mag-and-flag.  The  random 
(chance)  diagonal  for  Geophex  target  list  is  shown  as  a  dashed  line  in  Figure  16.  Random 
diagonals  for  beginning  to  the  end  of  NRL’s  and  NAEVA’s  target  list  can  also  be  drawn.  The 
high  initial  slope  of  the  NAEVA  ROC  curve  indicates  good  detection  and  discrimination 
(comparable  to  NRL,  better  than  Geophex,  and  considerably  better  than  mag-and-flag  across  all 
three  grids).  The  NAEVA  EM63  results  failed  to  reach  100%  detection  at  any  of  the  three  grids 
because  the  gate  3  threshold  was  set  conservatively  at  5  mV.  The  detection  would  probably  have 
reached  100%  at  a  4  mV  threshold  (as  shown  for  Grid  3  in  the  following  self-assessment 
discussion).  NAEVA’s  Pd  performance  for  all  three  grids  meets  Kaho’olawe  requirements  of 
85%  with  a  FAC  of  approximately  65  for  Grid  1,  130  for  Grid  2,  and  70  for  Grid  3. 
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JPG  V  On-site  Processing  Results  -  Area  2 


- NAEVA  - NRL  - Geophex 

-  -  -  Geophex  Random  Diagonal  ♦  MAG  &  Flag  Kaho  Pd  Requirement 


JPG  V  On  Site  Processing  Results  -  Area  3 


NAEVA  NRL  Geophex 

■  ■  ■  Geophex  Random  Diagonal  ♦  Mag  &  Flag  Kaho  Pd  Requirement 


Figure  16.  Composite  ROC  Curves  for  JPG. 
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4.3.3  Performance  Analysis  of  Grid  3 


The  JPG  Grid  3  truth  table  has  been  released,  making  it  possible  to  evaluate  decay  curve 
detection  and  discrimination  as  a  function  of  target  size,  depth,  and  amplitude  response.  Results 
are  summarized  graphically  in  Figure  17.  All  Grid  3  ordnance  (red,  blue,  and  yellow  triangles) 
and  nonordnance  (small  uncolored  squares)  are  plotted  against  mass  (x-axis)  and  depth  (y-axis). 
Two  presumed  basalt  samples  (green  squares)  are  shown.  Four  EM63  gate  3  amplitude  contours 
(threshold  5,  10,  15,  and  25  mV)  are  also  shown.  These  contours  were  determined  by  gridding 
and  contouring  the  gate  3  amplitude  response  of  all  detected  items  that  were  emplaced. 
Ordnance  that  was  not  detected  (outside  the  5  mV  threshold)  is  highlighted  in  blue.  Ordnance 
which  was  correctly  identified  is  highlighted  in  yellow.  Ordnance  which  was  misidentified  by 
decay  curve  analysis  is  highlighted  in  red.  It  is  interesting  that  both  57  mm  projectiles  and 
almost  all  of  the  60  mm  projectiles  were  misidentified.  Most  of  the  60  mm  were  misidentified  as 
81  mm  or  152  mm.  Parametric  plots  of  decay  curve  shape  were  constructed  to  detennine  why 
they  were  difficult  to  identify. 


Mass  versus  Depth  from 
JPG  5  Grid  3  Truth  Table 


2.75*  105mm  152mm  155mm 


I 


Legend 

A  Ordnance  Correctly 
Classified 

A  Ordnance  Detected 

A  Ordnance  Not 
Detected 

■  Basalt 

■  Non  Ordnance 


Figure  17.  Contours  of  Gate  3  Amplitude  (mV),  Grid  3,  JPG. 


NAEVA  Geophysics  introduced  parametric  plots  of  decay  curve  shapes  in  the  JPG  IV 
demonstration.  Parametric  plots  display  the  shape  (regardless  of  depth  or  amplitude  response)  in 
terms  of  parametric  curves  based  on  independent,  dimensionless  ratio  parameters  (gate  18/gate  2 
is  the  late  decay  curve  shape  and  gate  10/gate  2  is  the  early  decay  curve  shape).  If  an  unknown 
target  plots  on  one  of  these  parametric  curves,  it  has  a  similar  decay.  If  parametric  curves  for 
two  ordnance  types  overlap,  they  can  not  be  distinguished  by  decay  shape.  Parametric  plots  or 
decay  curve  shape  for  several  of  the  ordnance  samples,  shown  in  Figure  18,  reveal  that  the  decay 
curve  shapes  for  certain  sample  ordnance  types  overlap  closely.  On  the  other  hand,  the  9  in  long 
60  mm  projectiles  have  very  distinct  decay  curves.  It  is  therefore  easy  to  confuse  57  mm  and  7 
in  60  mm  with  each  other,  and  also  with  the  81  mm  mortar,  152  mm  projectile,  and  the  3.5  in 
calibration  sphere,  especially  if  the  response  is  noisy  (object  relatively  deep  for  its  size). 
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Figure  18.  Parameter  Plots  for  Decay  Curve  Shape,  Various  Ordnance  Types. 


Fortunately,  this  similarity  of  decay  curves  for  different  ordnance  does  not  affect  the 
prioritization  of  the  target  list  because  targets  are  prioritized  if  they  fit  any  expected  ordnance 
type  (in  the  chi-squared  sense). 
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4.3.3. 1  Multisensor  Towed  Array  Detection  System  (MTADS)  Magnetic  Data 

MTADS  magnetic  data  was  provided  after  completion  of  the  exercise  and  submittal  of  on-site 
results.  The  purpose  was  to  use  the  MTADS  data  in  conjunction  with  NAEVA’s  EM63  data  for 
additional  postprocessing.  Six  Grid  3  MTADS  magnetic  anomalies  were  selected  by  visual 
inspection  (with  no  corresponding  EM63  response).  Only  one  of  these  corresponds  to  a  Grid  3 
truth  table  target  (3-110),  which  is  evidently  a  basalt  boulder.  The  other  five  correspond  to  no 
Grid  3  truth  table  target;  the  causes  of  these  anomalies  are  unknown  (but  these  are  almost 
certainly  real  objects).  The  other  Grid  3  truth  table  target  that  is  evidently  basalt  (3-108) 
correlates  with  no  MTADS  or  EM  response.  However,  there  is  an  MTADS  and  a  weak  EM 
anomaly  approximately  2.5  m  to  the  east  that  is  unexplained.  Perhaps  item  3-108  is  mislocated  in 
the  Grid  3  truth  table. 

4.4  KAHO’OLAWE 

4.4.1  Objectives 

NAEVA’s  primary  technical  objective  at  the  Kaho’olawe  Controlled  Site  Demonstration  was  to 
determine  the  detection  and  discrimination  capabilities  of  the  EM63  multichannel,  time-domain 
EMI  system  in  a  high  magnetic  environment.  However,  to  achieve  this  main  goal,  a  number  of 
other  objectives  had  to  be  accomplished  during  the  test,  calibration,  and  revision-optimization 
period. 


•  Determine  whether  the  high  background  response  of  basaltic  materials  could  be  removed 
(by  autoleveling)  and/or  incorporated  into  the  chi-squared  fitting  procedures  (by  use  of  a 
characteristic  basalt  decay  vector).  To  determine  this,  the  following  tasks  had  to  be 
achieved: 

Evaluation  of  new  compensation  coil  and  other  noise  reduction  measures 

Measurement  and  evaluation  of  the  spatially  variable  background  response  of 
basalt  and  basaltic  soils  in  various  locations 

•  Enlarge  the  Decay  Curve  Analysis  Library 

•  Rewrite  algorithms  for  basalt  background 

4.4.2  Results/Analysis/Conclusions 

Demonstration  Grid  A  (four  30  m  x  30  m  subgrids)  contained  24  emplaced  ordnance  items,  28 
emplaced  nonordnance  items,  and  3  not  to  be  graded  (ng)  items,  according  to  the  final  ground 
truth  spread  sheet  (July  20,  2002).  In  addition,  18  digs  (investigations  of  unexplained 
demonstrator  targets)  were  reported,  of  which  11  were  significant  (had  separate  locations, 
productive  of  a  metal  object). 

Demonstration  Grid  B  (1  hectare,  12  subgrids)  contained  81  emplaced  ordnance  items,  90 
emplaced  nonordnance  items,  and  17  ng,  according  to  the  final  ground  truth  spreadsheet  (July 
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20,  2002).  In  addition,  61  digs  (investigations  of  unexplained  demonstrator  targets)  were 
reported,  of  which  34  were  significant  (had  separate  locations,  productive  of  a  metal  object). 

Demonstration  Grid  C  (six  30  m  x  30  m  subgrids)  contained  34  emplaced  ordnance  items,  40 
emplaced  non-ordnance  items,  and  1 1  ng  items,  according  to  the  final  ground  truth  spreadsheet 
(July  20,  2002).  In  addition,  57  digs  (investigations  of  unexplained  demonstrator  targets)  were 
reported,  of  which  40  were  significant  (had  separate  locations,  productive  of  a  metal  object). 

Our  self-evaluation  software  must  input  a  truth  table  with  classification,  mass,  and  depth 
information,  and  a  target  list.  It  produces  ROC  curves,  a  plan  map,  a  mass-depth  diagram 
(showing  detection  or  not),  and  a  histogram  of  location  errors  (up  to  the  detection  search  radius). 
Mass  information  was  not  provided  in  the  final  (May  20,  2002)  ground  truth  spread  sheet.  Some 
masses  are  estimates;  if  the  mass  of  an  item  is  not  known,  it  is  arbitrarily  assigned  as  1.0  kg. 

The  mass-depth  diagram  (Grid  A-Figure  19,  Grid  B-Figure  20,  and  Grid  C-Figure  21)  is 
probably  the  most  useful  display  of  results  because  it  is  possible  to  draw  an  approximate  limit- 
of-detection  line  on  the  diagram  that  can  be  compared  to  other  demonstrators  or  other  sites.  The 
diagram  shows  the  (estimated)  masses  and  depths  chosen  for  emplaced  items.  It  may  be  seen 
that  the  Kaho’olawe  detection  (using  basalt  discrimination)  approaches  JPG  detection,  which  is 
very  satisfying  considering  the  very  large  basaltic  soil  response.  Detection  limit  on  the  mass- 
depth  diagram  also  provides  information  needed  for  remediation  planning  and  quality  assurance 
(QA)  of  UXO  remediation. 


Figure  19.  Mass  Versus  Depth  Plot  for  Demo  Area  A,  Kaho’olawe. 
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Figure  20.  Mass  Versus  Depth  Plot  for  Demo  Area  B,  Kaho’olawe. 


The  ROC  curves  for  Demo  Areas  A,  B,  and  C,  respectively  (Figure  22,  Figure  23,  and  Figure  24) 
are  dependent  on  demonstrator  merit,  site  noise,  and  test  item  emplacement.  Percent  true 
positives  depend  on  the  emplacement  depth  chosen  (as  a  function  of  item  size  or  mass). 
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Figure  22.  ROC  Curve  for  Demo  Area  A,  Kaho’olawe. 

[Red  ROC  =  all  Ordnance,  green  ROC  =  Ordnance  without  20  mm, 
blue  ROC  =  all  emplaced  items  (all  ordnance,  including  20  mm,  and  nonordnance)] 


Figure  23.  ROC  Curve  for  Demo  Area  B,  Kaho’olawe. 

[Red  ROC  =  all  Ordnance,  green  ROC  =  Ordnance  without  20  mm, 
blue  ROC  =  all  emplaced  items  (all  ordnance,  including  20  mm,  and  nonordnance)] 
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Figure  24.  ROC  Curve  for  Demo  Area  C,  Kaho’olawe. 

[Red  ROC  =  all  Ordnance,  green  ROC  =  Ordnance  without  20  mm, 
blue  ROC  =  all  emplaced  items  (all  ordnance,  including  20  mm,  and  non-ordnance)] 

It  should  be  recognized  that  a  large  number  of  detected  targets  in  Demo  Grid  A  (more  than  53  of 
the  first  100),  Grid  B  (more  than  28  of  the  first  100),  and  Grid  C  (more  than  34  of  the  first  100) 
prioritized  target  list  do  not  correspond  to  any  emplaced  or  dug  items.  These  are  thought  to  be 
actual  metal  objects  that  were  not  detected  and  cleared  with  the  conventional  technology.  These 
targets  are  treated  as  “false  positives”  in  the  ROC  curves.  For  this  reason,  a  more  sensitive 
technology  (better  detection  of  metal  in  basalt)  will  have  a  correspondingly  worse  ROC  curve 
(more  false  positives). 

It  appears  that  NAEVA’s  location  errors  were  acceptable  based  on  the  location  error  histogram 
(Figure  25)  of  numerous  iron  rebar  rods  located  on  the  comers  of  the  30  m  x  30  m  subgrids.  The 
mode  of  these  errors  is  approximately  0.25  m  and  no  mislocations  greater  than  0.5  m. 
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Kaho-olawe, Histogram  of  Target  Location  Errors  of  Rebar  on  Sub-Grid  Comers 


Location  Errors  [meters] 


Figure  25.  Grid  Corner  Location  Error  Histogram  for  all  Demo  Areas,  Kaho’olawe  (comparing 

NAEVA’s  target  locations  to  grid  comers). 


Assuming,  based  on  the  acceptable  location  error  histogram  explained  above,  that  NAEVA’s 
locations  are  correct,  Figures  26  (Area  A),  Figure  27  (Area  B),  and  Figure  28  (Area  C)  (location 
error  histogram  comparing  NAEVA  targets  and  the  emplaced  targets)  reveal  that  a  substantial 
proportion  of  the  emplaced  items  exhibit  location  errors  between  0.5  m  and  1.5  m.  These 
location  errors  are  not  due  to  target  location,  which  is  good  to  20-40  cm.  Of  course,  these 
targets  will  not  show  up  as  detected  in  an  ROC  curve  with  a  0.5  m  search  radius,  giving  a 
misleading  impression  of  “%  true  positives.” 


Figure  26.  Emplaced  Item  Location  Error  Histogram  for  Demo  Area  A,  Kaho’olawe  (comparing 
NAEVA’s  target  location  to  the  location  of  the  emplaced  items). 
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Figure  27.  Emplaced  Item  Location  Error  Histogram  for  Demo  Area  B,  Kaho’olawe  (comparing 
NAEVA’s  target  location  to  the  location  of  the  emplaced  items). 


Figure  28.  Emplaced  Item  Location  Error  Histogram  for  Demo  Area  C,  Kaho’olawe  (comparing 
NAEVA’s  target  locations  to  the  location  of  the  emplaced  items). 
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5.0 


COST  ASSESSMENT 


5.1  COST  REPORTING 

5.1.1  Actual  Demonstration  Costs 

The  actual  costs  incurred  for  the  entire  ESTCP  Contract  Number  200035,  EM63  Decay  Curve 
Analysis  for  UXO  Discrimination,  was  $361,704.  The  breakdown  of  these  costs  by  tasks  is 
outlined  in  Table  2. 


Table  2.  Actual  Demonstration  Costs. 


Phase  I 

Task  1 

Controlled  Site  Data  Acquisition  at  Blossom  Point 

$23,520 

Task  2 

Data  Processing,  Algorithm  Development,  Software  Modification 

30,800 

Task  3 

Controlled  Site  Demonstration  at  JPG 

57,740 

Task4A:  Final  Report  Preparation 

10,990 

Task4B:  JPG  Self-Assessment 

11,300 

Phase  II 

Task  5 

Static  Bench  Testing 

15,150 

Task  6 

Initial  Kaho’olawe  Data  Review 

16,400 

Task  7 

Kaho’olawe  Controlled  Site  Demonstration 

94,130 

Task  8 

Kaho’olawe  Site  Data  Processing 

15,000 

Task  9 

Final  Report  Preparation  and  IPR  Meetings 

31,594 

Task  10:  Software  Preparation 

15,480 

Task  11:  Kaho’olawe  Self-Assessment 

39,600 

TOTAL 

$361,704 

5.1.2  Costs  of  Real  World  Implementation 

The  following  table  presents  estimated  expected  operational  costs  for  the  demonstrated 
technology  when  implemented,  not  including  mobilization  and  demobilization  costs  and  local 
transportation.  A  typical  project  is  broken  down  into  three  operational  phases: 

•  Presurvey  site  visit 

•  Static  bench  testing 

•  Digital  geophysical  mapping 

Data  acquisition 

Data  processing  for  detection  and  discrimination 

Costs  are  calculated  based  on  an  average  daily  rate  and  increase  or  decrease  depending  on  the 
size  and  duration  of  any  specific  project. 
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5.1.2. 1  Pre-Survey  Site  Visit 

Table  3.  Presurvey  Site  Visit  Costs. 


Item 

Cost 

Data  Acquisition 

Labor 

$1,350 

EM63  Equipment 

$  350 

GPS 

$  250 

Per  Diem 

$  240 

Materials 

$  100 

Data  Processing 

Labor 

$1,040 

Software 

$  100 

Materials 

$  40 

Data  Presentation 

Labor 

$  130 

Materials 

$  10 

TOTAL  (Daily  Rate) 

$3,610 

It  is  estimated  that  a  typical  project  may  require  a  presurvey  site  visit  of  5  days. 

5.1.2.2  Static  Bench  Testing 


Table  4.  Static  Bench  Testing  Costs. 


Item 

Cost 

Data  Acquisition 

Labor 

$1,350 

EM63  Equipment 

$  350 

Materials 

$  100 

Data  Processing 

Labor 

$1,040 

Software 

$  100 

Materials 

$  40 

Data  Presentation 

Labor 

$  130 

Materials 

$  10 

TOTAL  (Daily  Rate) 

$3,120 

It  is  estimated  that  a  typical  project  may  require  static  bench  testing  of  the  anticipated  ordnance 
items  for  about  5  days. 
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5.1.2.3  Digital  Geophysical  Mapping 


Table  5.  Digital  Geophysical  Mapping  Costs. 


Item 

Cost 

Data  Acquisition 

Labor 

$1,350 

EM63  Equipment 

$  350 

GPS 

$  250 

Per  Diem 

$  240 

Materials 

$  100 

Data  Processing 

Labor 

$1,040 

Software 

$  100 

Materials 

$  40 

Data  Presentation 

Labor 

$  130 

Materials 

$  10 

TOTAL  (Daily  Rate) 

$3,610 

It  is  estimated  that  a  two-person  field  crew  can  collect  approximately  Vi  to  P/2  acres  of  survey 

data  per  day,  depending  on  field  conditions. 

5.2  COST  ANALYSIS 

The  following  comments  are  offered  to  support  the  daily  unit  cost  per  item  presented  in  Tables  3, 

4,  and  5. 

•  Labor  (Data  Acquisition):  This  rate  reflects  average  state-of-the-industry  costs  for  a  two- 
person  geophysical  field  team  for  a  10-hour  workday. 

•  EM63:  This  reflects  a  typical  daily  use  fee  a  geophysical  subcontractor  would  charge  for 
use  of  an  EM63  instrument  in  the  field.  Such  a  fee  would  include  all  repair  and 
maintenance  costs,  as  well  as  equipment  replacement,  when  the  instrument  becomes 
obsolete  or  unreliable. 

•  GPS:  This  rate  reflects  a  typical  daily  use  fee  that  a  geophysical  subcontractor  would 
charge  for  use  of  a  GPS  system  (rover  and  base  station)  in  the  field,  and  the  software 
necessary  to  integrate  the  GPS  data  with  the  EM63  geophysical  data.  Such  a  fee  would 
include  all  repair  and  maintenance  costs,  as  well  as  equipment  replacement,  when  the 
instrument  becomes  obsolete  or  unreliable. 

•  Per  Diem:  This  rate  is  presented  as  an  average  per  diem  rate  for  an  average  project. 
Actual  per  diem  rates  would  be  those  presented  in  the  government  joint  travel  regulations 
(JTR). 

•  Materials:  An  average  cost  of  materials  is  calculated  based  on  $10.00  per  field  crew 
labor  hour  and  $5.00  per  labor  hour  of  data  processing  and  data  presentation. 
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•  Labor  (Data  Processing):  This  rate  reflects  average  state-of-the-industry  costs  for  16 
hours  of  data  processing  per  field  day  of  data  acquisition. 

•  Software:  This  rate  reflects  a  suggested  daily  fee  for  lease  of  the  EM63  data  processing 
software  for  detection  and  discrimination,  per  field  team. 

•  Labor  (Data  Presentation):  This  rate  reflects  average  state-of-the-industry  costs  for  2 
hours  of  data  presentation  per  field  day  of  data  acquisition. 


5.3  COST  COMPARISON  TO  CONVENTIONAL  TECHNOLOGY 

Table  6  presents  a  daily  cost  comparison  of  the  demonstrated  EM63  technology  to  conventional 
technology. 


Table  6.  Cost  Comparison  to  Conventional  Technology. 


Item 

EM63 

EM61 

Difference 

Data  Acquisition 

Labor 

$1,350 

$1,350 

$  0 

EM63  Equipment 

$  350 

$  250 

$  100 

GPS 

$  250 

$  250 

$  0 

Per  Diem 

$  240 

$  240 

$  0 

Materials 

$  100 

$  100 

$  0 

Data  Processing 

Labor 

$1,040 

$  520 

$  520 

Software 

$  100 

$  0 

$  100 

Materials 

$  40 

$  40 

$  0 

Data  Presentation 

Labor 

$  130 

$  130 

$  0 

Materials 

$  10 

$  10 

$  0 

TOTAL 

$3,610 

$,2890 

$  720 

The  cost  matrix  above  indicates  an  estimated  increase  in  daily  operating  costs  of  $720/day  for 
the  demonstrated  EM63  technology  over  EM61  technology.  This  increase  would  be  more  than 
offset,  however,  if  the  EM63  was  even  partially  successful  at  discriminating  true  UXO  targets 
from  clutter,  thus  reducing  the  number  of  UXO  excavations.  NAEVA  has  limited  exposure  to 
costs  for  UXO  excavations,  but  a  figure  provided  by  ESTCP  was  $200  per  dig.  Thus, 
eliminating  just  four  digs  per  day  would  offset  the  additional  costs  of  the  demonstrated  EM63 
technology.  In  practice,  if  an  “average”  daily  EM63  survey  produced  an  average  of  100  targets, 
and  the  demonstrated  technology  was  able  to  eliminate  only  18%  of  those  targets  from 
excavation,  the  cost  savings  would  approximate  the  entire  cost  of  conducting  the  digital 
geophysical  mapping  and  discrimination.  Higher  levels  of  successful  discrimination  would  yield 
potentially  huge  cost  savings. 

Specific  to  Kaho’olawe,  the  government  reports,  “As  of  1  March  2000,  contractors  at 
Kaho’olawe  had  detected  12,121  subsurface  anomalies  and  after  digging  they  found  that  only  4 
percent  are  UXO,  32  percent  are  false  positives  due  to  geologic  variations  and  64  percent  are  due 
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to  buried  metal  from  both  UXO  and  non-UXO-related  materials.”  Using  these  figures,  even  if 
the  demonstrated  technology  was  able  only  to  discriminate  metal  objects  (both  UXO  and  non- 
UXO)  from  magnetic  rocks/soil,  32%  of  target  excavations,  more  than  3,800  targets  would  have 
been  eliminated  with  a  savings  of  more  than  $760,000  at  a  minimum  (using  the  $200  cost  per  dig 
figure  from  ESTOP). 

Comparisons  should  also  be  made  to  mag-and-flag  detection  technology  for  which  NAEVA  does 
not  have  cost  figures.  Recent  results  from  ESTCP’s  JPG  2000  report  indicate  mag-and-flag 
detection  percentages  of  only  65  to  70  percent,  and  a  very  high  false  alarm  count.  Thus,  the 
demonstrated  EM63  technology  should  have  huge  cost  and  performance  advantages  over  mag- 
and-flag  technology. 
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6.0  IMPLEMENTATION  ISSUES 


6.1  COST  OBSERVATIONS 

Factors  that  affect  project  costs  include,  but  are  not  limited  to,  the  following: 

•  Project  location 

•  Project  size 

•  Terrain  conditions 

•  Vegetation  conditions 

•  Geologic  terrain  noise 

•  Scheduling  requirements 

6.1.1  Project  Location 

Project  location  affects  mobilization  and  demobilization  costs,  and  logistical  support  costs,  such 
as  shipment  of  spare  parts,  field  crew  break  travel  costs,  and  perhaps  data  transfer  costs. 

6.1.2  Project  Size 

Obviously,  the  larger  the  area  to  be  surveyed,  the  longer  it  will  take,  and  the  more  it  will  cost. 
Costs  are  not  necessarily  linear  with  acreage,  however,  as  site-specific  conditions  within  a 
project  area  may  drastically  affect  costs  per  acre.  Surveying  5  acres  of  difficult  topographic 
terrain  may  be  many  times  more  costly  than  twice  as  much  acreage  of  open  flat  terrain. 

6.1.3  Terrain  Conditions 

Terrain  conditions  affect  the  speed  at  which  a  survey  can  precede  and  therefore  how  much 
ground  can  be  covered  per  unit  of  time.  A  large  area  of  flat  open  terrain  could  be  surveyed  with 
a  vehicular  towed-array  system.  Ground  with  moderate  terrain  conditions  can  be  surveyed  with 
a  man-portable,  wheel-mode  cart  system,  while  rugged  terrain  may  have  to  be  surveyed  using  a 
dual  man  portable  “stretcher”  configuration.  Failure  to  implement  the  appropriate  sensor 
transport  configuration  may  compromise  data  quality.  Surveying  too  fast  may  lengthen  data 
point  spacing  beyond  acceptable  limits.  Surveying  in  wheel-mode  in  rough  terrain  induces  noise 
and  “spikes”  into  the  data.  Sensor  coils  need  to  be  kept  parallel  with  and  at  a  constant  distance 
from  the  ground  surface. 

6.1.4  Vegetation  Conditions 

Surface  vegetation  conditions  can  have  a  major  impact  on  project  costs.  Even  moderately 
wooded  terrain  prohibits  use  of  the  EM63  instrument.  Cleared  woods  allow  use  of  the 
“stretcher”  mode  configuration,  and  open  ground  (fields)  permits  use  of  the  man-portable  or 
vehicular-towed  configuration.  Vegetation  clearance  can  be  very  time-consuming  and 
expensive,  sometimes  approaching  or  even  exceeding  the  cost  of  the  geophysical  mapping. 
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6.1.5  Geologic  Terrain  Noise 


Geologic  terrain  noise  is  the  influence  of  the  bedrock  (and/or  overburden)  on  the  background 
electromagnetic  amplitude  response.  High  geologic  terrain  noise  complicates,  or  even  prohibits, 
valid  data  interpretation  of  standard  EM61  data.  Using  standard  EM61  survey  techniques  in 
geologic  noisy  terrain  causes  a  significant  increase  in  false  alarms  and  a  decrease  in  successful 
percent  detection.  One  of  the  great  successes  of  our  EM63  demonstration  at  Kaho’olawe  was  the 
ability  to  distinguish  buried  metal  targets  amidst  very  high  geologic  terrain  conditions.  Such 
ability  would  drastically  reduce  false  alarms  and  therefore  reduce  costs.  Generally,  however,  the 
lower  the  geologic  terrain  noise,  the  less  complicated  the  data  interpretation  will  be  and  the  less 
costly  the  survey. 

6.1.6  Scheduling  Requirements 

Scheduling  requirements  can  impact  project  costs.  Rush  jobs  tend  to  have  increased 
mobilization  costs  (higher  shipping  fees  and  travel  costs).  Sometimes  a  project  (or  client) 
requires  very  fast  data  processing  turnaround  time.  This  may  necessitate  implementation  of  an 
on-site  processor,  thus  incurring  higher  costs  for  travel  and  living  expenses. 

6.1.7  Potential  Cost  Reductions/Cost  Improvements 

The  best  approach  to  keeping  costs  low  is  through  comprehensive  advanced  project  planning. 
Adequate  scheduling  notice  should  be  implemented  to  keep  travel  and  shipping  costs  down. 
Presurvey  site  visits  should  be  conducted  to  assess  site-specific  project  conditions  and  objectives. 
Establishment  of  background  geologic  terrain  noise  should  be  accomplished  during  this  visit. 
Consensus  should  be  reached  as  to  vegetation  removal  techniques  that  would  optimize  digital 
geophysical  mapping.  Historical  research  should  be  conducted  to  ascertain  what  ordnance  types 
should  be  procured  and  made  available  to  the  contractor  for  measurement  prior  to  initiation  of 
field  activities. 

6.2  PERFORMANCE  OBSERVATIONS 

Refer  to  Section  4.2  -  Section  4.4. 

6.3  SCALE-UP 

A  concern  in  scale-up  to  a  full-scale  implementation  would  be  equipment  availability.  The 
EM63  instrument,  manufactured  by  Geonics  Ltd.  in  Mississauga,  Ontario,  is  not  a  mass- 
produced  instrument.  Only  four  or  five  have  been  manufactured  to  date,  each  one  with 
significant  modifications  from  previous  versions.  For  full-scale  implementation,  sufficient 
notice  should  be  given  to  the  manufacturer  to  produce  the  number  of  instruments  necessary  to 
equip  the  project  team(s).  Every  time  sensor  coil  geometry  and  parameters  are  changed, 
detection/discrimination  algorithms  must  be  modified,  and  samples  of  inert  ordnance  items  to  be 
discriminated  must  be  remeasured  to  reestablish  the  ordnance  signature  library  for  that  ordnance 
suite.  These  issues  may  be  mitigated  in  the  future  if  the  EM63  becomes  used  more  commonly 
and  its  identity  becomes  standardized. 
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6.4  OTHER  SIGNIFICANT  OBSERVATIONS 


It  is  critical  to  correctly  interpret  the  results  of  these  demonstrations  to  properly  evaluate  the 
benefits  and  limitation  of  this  discrimination  technology. 

6.5  LESSONS  LEARNED 

Problems  with  instrument  drift  and  instrument  noise  were  limitations  during  the  Kaho’olawe 
demonstrations.  Geonics  has  been  working  on  these  problems,  and  reports  (1)  reduction  in  noise 
caused  by  a  cooling  fan  in  the  instrument  console  and  (2)  software  modifications  for  correction 
of  early  channel  drift.  This  seems  promising,  but  NAEVA-GPA  cannot  verify  the  degree  of 
noise  and  drift  reduction  until  new  static  tests  are  made.  In  future  demonstrations  or  full-scale 
projects,  additional  time  for  testing  and  analysis  of  the  EM63  will  be  required  due  to  the 
continuing  development  by  Geonics. 

6.6  APPROACH  TO  REGULATORY  COMPLIANCE  AND  ACCEPTANCE 

The  primary  regulatory  issue,  which  will  affect  the  adoption  of  discrimination  technology  such 
as  EM63,  will  be  gaining  the  confidence  and  approval  of  federal,  state,  and  local  regulators, 
stakeholders,  and  users.  Acceptance  by  organizations  such  as  the  Army  Corps  of  Engineers  and 
Naval  Facilities  and  Engineering  Command  will  be  needed  in  order  that  future  RFPs  will  include 
such  innovative  technology.  This  controlled  site  ESTCP  demonstration  is  the  first  to  employ 
high  magnetic  (basaltic)  conditions,  which  will  allow  side-by-side  comparisons  of  discrimination 
performance,  production  rates,  and  costs.  Acceptance  of  discrimination  technology  (that  is,  not 
digging  some  of  a  prioritized  geophysical  target  list)  ultimately  requires  a  cost/risk  evaluation  by 
the  regulatory  agencies. 
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